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(54) Image processing apparatus and method. 

(57) Scenes having arbitrary sizes can be collec- 
tively edited. Dynamic image data Is managed 
by a tree structure in accordance with scenes. 
An array of scenes belonging to a higher order 
hierarchy than a reference hierarchy by one 
level is displayed in a scene display area (70a), 
an array of scenes belonging to the reference 
hierarchy is displayed in a scene display area 
(70b). and an array of scenes belonging to a 
given scene and having an order hierarchy 
lower by one level than the given scene which is 
designated by an operator and selected from 
scenes displayed in the reference hierarchy is 
displayed in a scene display area (70c). An 
arbitrary scene displayed in the scene display 
area (70b) is designated, and processing (copy, 
move, divide, and synthesize) for the designated 
scene is designated. A tree structure portion of 
the designated scene is modified in accordance 
with the designated processing. 
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BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to an image proc- 
essing apparatus and, more particularly, to an image 
processing apparatus and method suitable for editing 
a dynamic image. 

Related Background Arts 

In a known conventional arrangement, cuts con- 
stituting a dynamic image such as a video image are 
automatically or manually extracted from the dynamic 
image and rearranged along the time base in accor- 
dance with the reproduction order to edit the dynamic 
image. 

The cuts indicate dynamic image data groups 
serving as minimum constituent units of the dynamic 
image and constituted by image data divided by 
scene changes. A scene indicates an Image data 
group consisting of one or two or more cuts given a 
certain meaning. For this reason, the term "scene" 
may Indicate one or more scenes. 

Each extracted cut is represented by an icon (to 
be referred to as a cut icon hereinafter) consisting of 
a reduced representative image (normally a start 
frame) of the cut. The cut icon is displayed on a 
screen. A technique for manipulating this cut icon on 
the screen to edit a desired dynamic image is also 
available. For example, the order of cut icons is 
changed to change the reproduction order. According 
to this technique, dynamic image editing can be inter- 
rogatively performed while an edited dynamic image 
is obtained in red time. 

According to this method, cut icons whose num- 
ber is equal to the number of cuts are used, and dy- 
namic image editing Is a simple repetition of monot- 
onous operations for rearranging these cut icons. 
Therefore, when the number of cut icons increases, 
the number of operations for the cut icons Increases 
like a geometric series, thereby extremely complicat- 
ing the manipulations. 

In other words, a plurality of cuts cannot be edited 
at once. As described above, the number of repetitive, 
monotonous operations undesirably increases with 
an increase in the number of cuts. In addition to edit- 
ing in units of cuts, an editor may want to edit using 
a scenario, i.e., a plurality of cuts as a unit which is 
different from that of each cut. In addition to dynamic 
image editing in units of cuts, demand has arisen for 
providing an editing technique capable of arbitrarily 
setting and selecting the unit of editing in accordance 
with the will of the editor. 

SUMMARY OF THE INVENTION 

It is the primary object of the present invention to 



provide an image processing apparatus having an 
edit function for responding to the above demand. 

It is another object of the present Invention to pro- 
vide an image processing apparatus and method ca- 
5 pable of editing a dynamic image in an arbitrary unit 
of editing. 

In order to achieve the above objects of the pres- 
ent invention, there is provided an image processing 
apparatus for editing image data constituted by an 

10 Image data group of a plurality of frames. The Image 
apparatus comprises memory means for classifying 
the image data group into a plurality of sets in accor- 
dance with characteristics of the image data and stor- 
ing a relationship between the classified sets, display 

15 means for performing a hierarchical display of the 
sets of the image data in accordance with the rela- 
tionship stored in the memory means, and designat- 
ing means for designating editing of the image data 
group in units of the sets. 

20 It is still another object of the prese nt Invention to 

provide an image processing apparatus and method 
capable of allowing a user to easily perform dynamic 
image editing. 

it Is still another object of the present invention to 

25 provide an image processing apparatus and method 
capable of performing edit processing for each hier- 
archical level after a dynamic image is hierarchically 
defined. 

It is still another object of the present invention to 
30 provide an image processing apparatus and method 
having a novel function. 

The above and other objects, features, and ad- 
vantages of the present invention will be apparent 
from the following detailed description in conjunction 
35 with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram for explaining the prin- 
40 ciple of the function of an embodiment of the 

present invention; 

Fig. 2 is a view for explaining the hierarchical 
structure of a scene; 

Fig. 3 is a block diagram showing a hardware ar- 
45 rangement for realizing this embodiment; 

Fig. 4 is a plan view of a mouse 40; 
Fig. 5 is a view for explaining an edit screen of this 
embodiment; 

Fig. 6 is a view for explaining a scene display 
so screen; 

Fig. 7 is a view for explaining the relationship be- 
tween the edit screen shown In Fig. 5 and a tree 
structure; 

Fig. 8 is a flow chart showing part of the operation 
55 upon manipulations of the mouse 40; 

Fig. 9 is a flow chart showing the remaining part 
of the operation upon manipulations of the mouse 
40; 
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Fig. 10 is a view for explaining an edit command 
menu; 

Fig. 11 is a view for explaining a modification of 
the tree structure for a copy command; 
Fig. 12 is a view for explaining a modification of 
the tree structure for a delete command; 
Fig. 13 is a view for explaining a modification of 
the tree structure for a move command; 
Fig. 14 is a view for explaining a modification of 
the tree structure for a divide command; 
Fig. 15 is a view for explaining a modification of 
the tree structure for a synthesize command; 
Fig. 16 is a view showing a new edit screen; and 
Fig. 17 is a flow chart showing a case in which 
specific functions are assigned to mouse but- 
tons. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

A preferred embodiment of the present invention 
will be described in detail with reference to the accom- 
panying drawings. 

Fig. 1 Is a block diagram showing the schematic 
arrangement of a preferred embodiment of the pres- 
ent invention. Referring to Fig. 1, an input apparatus 
10 comprised of a mouse and a keyboard is manipu- 
lated by a user or operator. A determination unit 12 de- 
termines whether an input from the input apparatus 
10 represents a horizontal scroll 14, a vertical scroll 
15, a scene designation 18, or an edit command 20. 

When the input represents the horizontal scroll 
14 and the vertical scroll 15, a scene as the center of 
the display is registered in a reference pointer 22. 
When the input represents the scene designation 18, 
a scene as an edit manipulation target is registered in 
an access pointer 24. When the input represents the 
edit command 20, a scene having a lower order hier- 
archy than the scene registered in the access pointer 
24 is edited. The detailed contents of editing are de- 
scribed in detail in a subsequent explanation of the 
edit command. Various processing operations for 
modifying a tree structure are performed on the basis 
of designated edit commands, and processing results 
are registered in a scene structure management unit 
26, so that arbitrary editing can be performed for a 
scene structure, thereby allowing editing of dynamic 
Image data. A display control unit 28 refers to the 
scene structure management unit 26 and the refer- 
ence pointer 22, and generates scene icons centered 
on a scene represented by the reference pointer 22 on 
the basis of the scene structure of the scene struc- 
ture management unit 26. The display control unit 28 
displays these scene icons on a monitor unit 30 in a 
manner to be described later in accordance with the 
scene structure. 

Hierarchically described dynamic image data in 
this embodiment will be described with reference to 



Fig. 2. 

Dynamic data consists of a series of still images 
continuous in the time direction. Aseries of screen im- 
age data can be divided into cuts or scenes in accor- 

5 dance with the contents of the screen image data. As 
previously described, each cut constitutes an image 
data group as a minimum constituent unit of the dy- 
namic image and is divided by scene changes. An im- 
age data group constituting a series of cuts having a 

10 certain meaning is called a scene. Each scene repre- 
sents all or part of the dynamic image depending on 
its meaning. That is, an image group in a baseball 
game is taken as an example of edit images in Fig. 2. 
When a scene is defined by a set of "ooo vs. xxx, 

15 Ninth Game", the scene becomes all the dynamic im- 
age (i.e., the entire image data group). When a scene 
is defined as a set of "Pitcher Up, First Inning, Ninth 
Game", the scene is represented by several cuts (or 
one cut). For this reason, one image data group can 

20 be managed by the data structure of a hierarchical 
tree structure using scenes and cuts, as shown in 
Fig. 2. A scene having the lowest order hierarchy is a 
cut, and the respective cuts correspond to the respec- 
tive parts of the image data group. A scene having a 

25 high order hierarchy consists of a combination of 
scenes having lower order hierarchies. A scene hav- 
ing the highest order hierarchy represents the entire 
dynamic image constituted by all the cuts. 

The image data group has an order in the time di- 

30 rection. In a tree structure, a left cut or scene is ear- 
lier than a right cut or scene in the time direction. 
When the position of a scene in the tree structure is 
changed, the order of cuts having lower order hierar- 
chies than that of this scene is changed, and the re- 

35 production order of cuts constituting the dynamic im- 
age is changed. That is, a dynamic image constituted 
by the image data group can be edited. 

Fig. 3 is a block diagram showing a hardware ar- 
rangement according to this embodiment. A mouse 40 

40 serves as a coordinate input apparatus for inputting 
coordinates on a monitor. A keyboard 42 is connected 
to a system bus 46 through an input control circuit 44. 
A large-capacity memory 48 for storing dynamic im- 
age data, a data base 50 for managing the dynamic 

45 image data In a hierarchical structure, a display con- 
trol unit 52 for controlling a display on the monitor 
screen, and a CPU 54 for controlling the overall ap- 
paratus and processing programs (to be described 
later) are also connected to the system bus 46. The 

50 display control circuit 52 displays a desired image on 
the screen of a monitor unit 58 through a display 
memory 56. 

A program memory 60 and a work memory 62 are 
connected to the CPU 54. The program memory 60 
55 stores a dynamic image edit program having a coor- 
dinate input processing routine and a dynamic image 
edit command processing routine. The work memory 
62 is used to temporarily store variables such as a 
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present cursor coordinate position and reference 
data. 

As shown in Fig. 4, the mouse 40 has three but- 
tons and outputs data representing moving amounts 
Ax and Ay and a signal representing a button manipu- s 
lated state. When a button of the mouse 40 is de- 
pressed, the mouse 40 outputs a depressed button 
number, #1, #2, or #3. When no button is depressed, 
the mouse 40 outputs #0. The moving amounts Ax 
and Ay of the mouse 40 and the button number of a 10 
depressed button are periodically input from the 
mouse 40 to the input control circuit 44. The input con- 
trol circuit 44 transfers the input moving amounts Ax 
and Ay and the input button number data to the CPU 
54 through the system bus 46. When the input button 15 
number is #0, the CPU 54 changes the cursor coor- 
dinates corresponding to the input moving amounts 
Ax and Ay. The CPU 54 stores the changed cursor co- 
ordinate data in the work memory 62 and outputs 
them to the display control circuit 52. 20 

The display control circuit 52 writes a cursor im- 
age at an address of the display memory 56 which re- 
flects the input cursor coordinate values. Therefore, 
the monitor unit 58 displays a cursor at a screen pos- 
ition correspondi ng to the movement of the mouse 40. 25 

The data base 50 manages the tree structure 
data and corresponds to the scene structure man- 
agement unit 26 in Fig. 1. A unit constituting the tree 
structure, i.e., a node corresponds to one scene of a 
dynamic image, and the corresponding dynamic im- 30 
age scene is designated by a specially assigned num- 
ber (ID number). In this data base, a plurality of tree 
structures can be managed. The dynamic image data 
is stored in the memory 48. Note that the data base 
50 stores address information corresponding to the ID 35 
number of each scene constituting the dynamic im- 
age stored in the memory 48 and position information 
in the tree structure of each scene. 

Fig. 5 shows an edit screen of this embodiment. 
Under the control of the CPU, scene designation in- 40 
formation and a scene adjacent to the designated 
scene in the tree structure are read out from the data 
base 50, and the memory 48 is accessed to allow a 
display of a desired scene on an edit screen. This edit 
screen is comprised of three horizontally elongated 45 
scene display areas 70a, 70b, and 70c for displaying 
scenes as icons, horizontal scroll bars 72a, 72b, and 
72c respectively located below the scene display 
areas 70a, 70b, and 70c and used to manipulate the 
display positions within the areas of the correspond- so 
ing scene display areas 70a, 70b, and 70c, and a vert- 
ical scroll bar 74, located at the right of the edit screen 
and used to change the hierarchy of data displayed in 
an active one of the three scene display areas 70a, 
70b, and 70c. That is, the vertical scroll bar 74 Is used 55 
to change the hierarchy in designating a scene to be 
edited. 

In each of the scene display areas 70a, 70b, and 



70c, scenes of a designated hierarchy are displayed 
in an array of icons (reduced images) representing 
the respective scenes centered on the node repre- 
sented by the reference pointer 22, as shown in Fig. 
6. An icon is displayed for each scene. When a pre- 
determined still image as a representative image of 
the scene which represents the contents of the scene 
is designated, the designated image is reduced into 
a predetermined size, and the reduced image is used 
as a scene icon. However, when no still Image is des- 
ignated, the first image is reduced into the predeter- 
mined size, and the reduced image is used as a scene 
icon. The presently referred scene icon may be sur- 
rounded by a thick frame or flickered to inform the edi- 
tor of present designation, which technique is known 
well to those skilled in the art. 

As shown in Fig. 7, the three scene display areas 
70a, 70b, and 70c display a scene structure having 
three different hierarchies. The scene display area 
70a displays a scene having a higher order hierarchy 
than that of the reference pointer 22 by one level. The 
scene display area 70b displays a scene having the 
same hierarchical level as that of the reference poin- 
ter 22. The scene display area 70c displays a scene 
having a lower order hierarchy than that of the refer- 
ence pointer 22 by one level and belonging to the 
node of the reference pointer 22. 

An operation upon depression of a button of the 
mouse 40 will be described below. Figs. 8 and 9 are 
flow charts showing the operation of the mouse 40. 
When the button number data from the mouse 40 rep- 
resents a number except for #0, the CPU 54 starts the 
processes of the flow charts shown in Figs. 8 and 9. 
First of all, the CPU 54 reads or obtains the present 
cursor coordinates from the work memory 62 (S1) 
and reads the button number of a button depressed 
on the mouse 40 (S2). 

When the button number Is "1" (S4), the CPU 54 
determines whether the present cursor coordinates 
fall within the area of the horizontal scroll bar 72a, 
72b, or 72c, the vertical scroll bar 74, or the scene dis- 
play area 70a, 70b, or 70c (S5, S6, and S7). 

When the cursor coordinates fall within the area 
of the horizontal scroll bar 72a, 72b, or 72c (S5), the 
relative coordinate values within the corresponding 
horizontal scroll bar are calculated, and a correspond- 
ing node within the same hierarchical level is regis- 
tered as a new reference pointer (SB). Therefore, in 
Fig. 2, when the reference pointer is, for example, 
scene 11, the horizontal scroll bar 72b is manipulated 
to allow changes in reference pointer within the same 
hierarchical level in an order of scene 11, scene 12 

If the present cursor coordinates fall within the 
area of the vertical scroll bar 74 (S6), the correspond- 
ing hierarchy is calculated from the coordinates in the 
vertical scroll bar 74. If the calculated hierarchy has 
a higher order than that of the present reference point, 
a node (constituent element of the tree structure) In- 
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eluding the present reference point within this higher 
order hierarchy is recorded in the reference pointer 22 
as a new reference point. That is. If the present refer- 
ence pointer represents scene 11 in Fig. 7, the upper 
portion of the vertical scroll bar is designated to 5 
change the reference pointer to scene 1 . However, if 
the calculated hierarchy has a lower order than that 
of the present reference point, the start node of ail the 
nodes belonging to the presently referred point is re- 
corded In the reference pointer 22 as a new pointer 10 
(S9). That is, if the present reference pointer repre- 
sents scene 11 in Fig. 7, the lower portion of the vert- 
ical scroll bar is designated to change the reference 
pointer to scene 111 . When the new pointer is record- 
ed in the reference pointer 22, the display control unit 1 5 
28 displays an array of scenes having the changed 
reference point as the center on the scene display 
area 70a, 70b, or 70c on the screen of the monitor unit 
30. Therefore, when the reference pointer 22 repre- 
sents a higher order hierarchy, the display control unit 20 
28 displays the reference scene of the newly set high- 
er order hierarchy in the scene display area 70b in Fig. 
5. 

When the present cursor coordinates fall within 
the area of the scene display area 70a, 70b, or 70c 25 
(S7), and is located on any one of the scene icons, the 
ID number of the designated scene icon is recorded 
In the access pointer 24 (S10). 

Even if the button number is "1" (S4), if the pres- 
ent cursor coordinates fall within none of the area of 30 
the horizontal scroll bars 72a, 72b, and 72c, the vert- 
ical scroll bar 74, and the scene display areas 70a, 
70b, and 70c, the operation is ended. 

If the button number is "3" (S3), edit menu titles 
shown in Fig. 10 are displayed (S11). When any one 35 
of the items as the edit menu titles is selected (S12), 
an edit command corresponding to the selected item 
Is executed (S13 to S22). When an area except for the 
menu title area is designated, the operation is ended. 

In this embodiment edit commands are scene 40 
"copy", "delete", "move", "divide", and "synthesize" 
commands, as shown in Fig. 10. Processing opera- 
tions designated by these commands can be realized 
by actually manipulating tree structure data which hi- 
erarchically describes a dynamic image in question. 45 

Editing will be described with reference to actual 
edit operations. To designate a scene to be edited, 
horizontal and vertical scroll operations are per- 
formed in the scene display area 70b to display an 
icon of the edit target scene. Subsequently, the cur- so 
sor is moved onto the icon of the displayed scene, and 
button #1 of the mouse 40 is depressed to designate 
the edit target scene. After the edit target scene is se- 
lected, button #3 of the mouse 40 is depressed to dis- 
play the menu titles shown in Fig. 10. The cursor is 55 
moved onto one of the menu items, and button #1 of 
the mouse 1 is depressed to designate an edit com- 
mand for the edit target scene. If the designated com- 



mand is a copy command, the storage destination of 
the copied node is designated with the mouse 40. 
However, if the designated command is a move com- 
mand, the moving destination is designated with the 
mouse 40. 

Each edit command executes processing for 
modifying or rearranging general tree structure data. 
For this reason, processing for modifying the tree 
structure corresponding to each edit command will be 
described below. 

As shown in Fig. 11, the copy command creates 
a tree structure having the same structure as a tree 
structure having, as a vertex, a scene designated by 
the scene designation 18, and locates the created 
tree structure data at an arbitrarily designated loca- 
tion. For example, to repeatedly display a given scene 
in a dynamic image, this scene is repeatedly created 
by copy commands, and the resultant identical 
scenes are continuously arranged. 

As shown in Fig. 12, the delete command deletes 
a tree structure having, as a vertex, a scene desig- 
nated by the scene designation 1 8. This command is 
used to delete unnecessary scenes in a dynamic im- 
age. 

As shown in Fig. 13, the move command moves 
a tree structure having, as a vertex, a scene desig- 
nated by the scene designation 18 to an arbitrarily 
designated location. This command is used to change 
the order of scenes. 

As shown in Fig. 14, the divide command divides 
scenes into two parts using a scene designated by 
the scene designation 18 as a boundary. This com- 
mand is used when a scene having a high order hier- 
archy has an excessive number of scenes or cuts and 
must be divided into parts. 

As shown in Fig. 15, the synthesize command 
synthesizes two or more scenes designated by the 
scene designation 18 into one scene. This command 
is used to synthesize two scenes into one scene to 
edit a larger number of cuts or scenes at once. 

By the manipulations using the mouse on the 
screen, as described above, a tree structure which 
describes a dynamic image can be modified. The 
modified tree structure is stored in the data base 50. 
For this reason, the order of cuts having a lower order 
hierarchy in the modified tree structure is changed, 
and the dynamic image can be edited. In addition, a 
hierarchical structure is used to edit a dynamic image 
using a higher hierarchy, so that a plurality of cuts can 
be manipulated at once, thereby greatly reducing the 
manipulation load. 

In the above embodiment, one dynamic image In 
which a scene arrangement is described by a tree 
structure is edited by modifying the corresponding 
tree structure. However, a method of sequentially ex- 
tracting desired scenes from one or a plurality of dy- 
namic images and creating a new tree structure may 
be employed. 



5 



9 



EP 0 682 324 A2 



10 



In this case, the edit screen is a scene designa- 
tion screen for designating an arbitrary scene (node) 
in a dynamic image serving as an edit source. In ad- 
dition, an edit scene display area (Fig. 1 6) for arrang- 
ing scenes for constituting a newly created dynamic 
image (tree structure) is used. 

A scene used for editing upon the scene designa- 
tion operation described above is designated. The 
designated scene (node) is copied in the edit scene 
display area shown In Fig. 16. This operation Is equiv- 
alent to an operation in which the designated node is 
copied in tree structure data (edit) different from the 
tree structure data to which the designated node be- 
longs. All the scenes arranged in the edit scene dis- 
play area belong to the edit tree structure data. By re- 
peating this operation, new tree structure data is cre- 
ated. Since the edit command is a command for mod- 
ifying a tree structure, this command is similarly ef- 
fective for the newly created tree structure. This 
method is effective in creating a digest by acquiring 
the respective parts of long-span dynamic image 
data. 

It is also possible not to extract partial trees con- 
stituting new target tree structure data from one tree 
structure data, but from a plurality of tree structure 
data (a plurality of dynamic images). In this case, a 
switch for switching between tree structures to be 
displayed on the edit screen shown in Fig. 5 is ar- 
ranged, and part of an arbitrarily designated tree 
structure data is designated and copied in the edit 
scene display area. 

When a change in display hierarchy is assigned 
to a specific button of the mouse 50 or a specific key 
of the keyboard 42, dynamic image editing can be 
performed without using scroll bars. This is a known 
technique in the field of computer software as a so- 
called short-cut 

Fig. 17 Is a flow chart for assigning specific func- 
tions to the three buttons of the mouse 40 depending 
on given situations. For example, when the cursor is 
located on a scene icon in a single scene display area, 
as shown in Fig. 6, and button #3 of the mouse 40 is 
depressed (S32 and S35), the reference pointer 22 is 
changed to a scene (node) having an order hierarchy 
higher by one level, and an array of scene icons each 
having an order hierarchy higher by one level is dis- 
played in the scene display area (S38). When button 
#2 is depressed (S33 and S36), the reference pointer 
22 is changed to a scene (node) having an order hi- 
erarchy lower by one level, and an array of scene 
icons each having an order hierarchy lower by one 
level Is displayed in the scene display area (S40). 

When the cursor is located at a position except for 
the scene icons within the scene display area, and 
button #3 is depressed (S32 and S35), the reference 
pointer 22 is moved to a right scene (icon) by the num- 
ber of scene icons displayed in the present scene dis- 
play area, and an array of scene icons located to the 



right of the presently referred scene in the scene dis- 
play area is displayed (S39). When button #2 is de- 
pressed (S33 and S36), an array of left scene Icons 
is displayed (S41). 

5 When the cursor is located on a scene icon, and 

button #1 is depressed (S34 and S37), data for dis- 
playing the input edit command menu is input to the 
access pointer 24, and a menu for selecting and edit- 
ing scenes is displayed (S42). An edit command can 

10 be executed In the same manner as in the above em- 
bodiment 

The tree structure of this embodiment is not lim- 
ited to the one displayed in Fig. 7, and a tree structure 
in another form may be used, as a matter of course. 

is To manually input data to modify a tree structure, a 
touch panel on a screen or any other means may be 
used in addition to use of the mouse as in this embodi- 
ment as far as the data can be manually input. 

In this embodiment, the scene structure is dis- 

20 played on the display screen shown in Fig. 5 in the 
form of a tree structure. However, the scene struc- 
ture is not limited to this. For example, tree structures 
shown in Figs. 11 to 15 may be displayed without any 
modifications, and these tree structures may be 

25 modified on the display screen. With this arrange- 
ment, a tree structure can be simply displayed. Note 
that a method of displaying the edited dynamic image 
data Is to access dynamic Images in the memory 48 
under the control of the CPU on the basis of the tree 

30 structure modified and stored in the data base and to 
sequentially display them on the monitor unit 48. 

As can be readily understood from the above de- 
scription, according to the embodiment of the present 
invention, a dynamic Image In which a scene struc- 

35 ture is hierarchically described can be edited by mod- 
ifying the corresponding tree structure. Therefore, 
one or more cuts or scenes can be collectively and hi- 
erarchically processed, and the number of manipula- 
tions can be greatly reduced, thereby greatly improv- 

40 ing operability. 



Claims 

45 1 . An image processing apparatus for editing image 
data constituted by an image data group of a plur- 
ality of frames, comprising: 

memory means for classifying the image 
data group into a plurality of sets in accordance 
so with characteristics of the image data and storing 

a relationship between the classified sets; 

display means for performing a hierarchi- 
cal display of the sets of the image data in accor- 
dance with the relationship stored in said mem- 
55 ory means; and 

designating means for designating editing 
of the image data group in units of the sets. 
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2. An apparatus according to claim 1, wherein the 
image data group is stored in a memory medium. 

3. An apparatus according to claim 1, wherein the 
set comprises at least one frame image. 

4. An apparatus according to claim 1, wherein the 
hierarchical display is of a tree structure. 

5. An apparatus according to claim 1, wherein said 
display means performs a multi-image display of 
representative image data of the sets. 

6. An apparatus according to claim 5, wherein the 
representative image data is a start frame image 
of the sets. 



editing is processing for deleting a set designated 
by said designating means from the image data 
group. 

5 15. An apparatus according to claim 1, wherein the 
editing is processing for moving a set designated 
by said designating means in the image data 
group. 

10 16. An apparatus according to claim 1. wherein the 
editing is processing for dividing a set designated 
by said designating means into a plurality of sets. 

17. An apparatus according to claim 1, wherein the 
15 editing is processing for synthesizing a plurality 

of sets designated by said designating means. 



7. An apparatus according to claim 5, wherein the 
image data represents an image obtained by re- 
ducing a start frame image of the image data 
group. 



11. An apparatus according to claim 1, further com- 
prising edit menu display means for displaying an 
edit menu corresponding to contents of the edit- 
ing. 



18. An apparatus according to claim 1, wherein the 
hierarchical display is to simultaneously display a 

20 plurality of hierarchies. 

19. An apparatus according to claim 1, further com- 
prising: 

control means for changing the relation- 
ship between the sets in said memory means in 
accordance with the designation; and 

processing means for editing the image 
data group in accordance with the changed rela- 
tionship between the sets. 

Apparatus or a method for image processing in 
which image data for a plurality of image frames 
are arranged, for example displayed, hierarchi- 
cally in accordance with relationships between 
the image frames for enabling editing. 

21. Apparatus or a method according to claim 20, 
wherein the Image data are classified into a plur- 
ality of sets and the relationship is between sets 
40 of image data. 



8. An apparatus according to claim 1, wherein said 
display means simultaneously displays sets hav- 
ing different hierarchies. 25 

9. An apparatus according to claim 1, wherein said 
display means simultaneously displays sets hav- 
ing the same hierarchy. 

30 

10. An apparatus according to claim 1, further com- 20. 
prising changing means for changing the relation- 
ship between the sets in said memory means in 
accordance with a designation from said desig- 
nating means. 35 



12. An apparatus according to claim 1, wherein 

said display means comprises edit menu 
display means for displaying contents of the edit- 
ing, and 45 
said designating means comprises 
edit content designating means for desig- 
nating the contents of the editing on said edit 
menu display means, and 

set designating means for designating edit so 
target sets of the sets displayed on said display 
means. 



13. An apparatus according to claim 1, wherein the 

editing Is processing for copying a set designated 55 
by said designating means. 



14. An apparatus according to claim 1, wherein the 
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(54) Image processing apparatus and method 

(57) Scenes having arbitrary sizes can be collective- 
ly edited. Dynamic image data is managed by a tree 
structure in accordance with scenes. An array of scenes 
belonging to a higher order hierarchy than a reference 
hierarchy by one level is displayed in a scene display 
area (70a), an array of scenes belonging to the refer- 
ence hierarchy is displayed in a scene display area 
(70b), and an array of scenes belonging to a given scene 
and having an order hierarchy lower by one level than 
the given scene which is designated by an operator and 
selected from scenes displayed in the reference hierar- 
chy is displayed in a scene display area (70c). An arbi- 
trary scene displayed in the scene display area (70b) is 
designated, and processing (copy, move, divide, and 
synthesize) for the designated scene is designated. A 
tree structure portion of the designated scene is modi- 
fied in accordance with the designated processing. 
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